Deterministic Versus Randomized Kaczmarz Iterative Projection

Kaczmarz's alternating projection method has been widely used for solving a consistent (mostly over-determined) linear system of equations Ax=b. Because of its simple iterative nature with light computation, this method was successfully applied in computerized tomography. Since tomography generates a matrix A with highly coherent rows, randomized Kaczmarz algorithm is expected to provide faster convergence as it picks a row for each iteration at random, based on a certain probability distribution. It was recently shown that picking a row at random, proportional with its norm, makes the iteration converge exponentially in expectation with a decay constant that depends on the scaled condition number of A and not the number of equations. Since Kaczmarz's method is a subspace projection method, the convergence rate for simple Kaczmarz algorithm was developed in terms of subspace angles. This paper provides analyses of simple and randomized Kaczmarz algorithms and explain the link between them. It also propose new versions of randomization that may speed up convergence

Nearness to Local Subspace Algorithm for Subspace and Motion Segmentation

There is a growing interest in computer science, engineering, and mathematics for modeling signals in terms of union of subspaces and manifolds. Subspace segmentation and clustering of high dimensional data drawn from a union of subspaces are especially important with many practical applications in computer vision, image and signal processing, communications, and information theory. This paper presents a clustering algorithm for high dimensional data that comes from a union of lower dimensional subspaces of equal and known dimensions. Such cases occur in many data clustering problems, such as motion segmentation and face recognition. The algorithm is reliable in the presence of noise, and applied to the Hopkins 155 Dataset, it generates the best results to date for motion segmentation. The two motion, three motion, and overall segmentation rates for the video sequences are 99.43%, 98.69%, and 99.24%, respectively

Subspace Segmentation And High-Dimensional Data Analysis

This thesis developed theory and associated algorithms to solve subspace segmentation problem. Given a set of data W={w_1,...,w_N} in R^D that comes from a union of subspaces, we focused on determining a nonlinear model of the form U={S_i}_{i in I}, where S_i is a set of subspaces, that is nearest to W. The model is then used to classify W into clusters. Our first approach is based on the binary reduced row echelon form of data matrix. We prove that, in absence of noise, our approach can find the number of subspaces, their dimensions, and an orthonormal basis for each subspace S_i. We provide a comprehensive analysis of our theory and determine its limitations and strengths in presence of outliers and noise. Our second approach is based on nearness to local subspaces approach and it can handle noise effectively, but it works only in special cases of the general subspace segmentation problem (i.e., subspaces of equal and known dimensions). Our approach is based on the computation of a binary similarity matrix for the data points. A local subspace is first estimated for each data point. Then, a distance matrix is generated by computing the distances between the local subspaces and points. The distance matrix is converted to the similarity matrix by applying a data-driven threshold. The problem is then transformed to segmentation of subspaces of dimension 1 instead of subspaces of dimension d. The algorithm was applied to the Hopkins 155 Dataset and generated the best results to date

CUR Decompositions, Similarity Matrices, and Subspace Clustering

A general framework for solving the subspace clustering problem using the CUR decomposition is presented. The CUR decomposition provides a natural way to construct similarity matrices for data that come from a union of unknown subspaces $\mathscr{U}=\underset{i=1}{\overset{M}\bigcup}S_i$. The similarity matrices thus constructed give the exact clustering in the noise-free case. Additionally, this decomposition gives rise to many distinct similarity matrices from a given set of data, which allow enough flexibility to perform accurate clustering of noisy data. We also show that two known methods for subspace clustering can be derived from the CUR decomposition. An algorithm based on the theoretical construction of similarity matrices is presented, and experiments on synthetic and real data are presented to test the method. Additionally, an adaptation of our CUR based similarity matrices is utilized to provide a heuristic algorithm for subspace clustering; this algorithm yields the best overall performance to date for clustering the Hopkins155 motion segmentation dataset.Comment: Approximately 30 pages. Current version contains improved algorithm and numerical experiments from the previous versio

Reduced row echelon form and non-linear approximation for subspace segmentation and high-dimensional data clustering

Given a set of data W={w1,…,wN}∈RD drawn from a union of subspaces, we focus on determining a nonlinear model of the form U=⋃i∈ISi, where {Si⊂RD}i∈I is a set of subspaces, that is nearest to W. The model is then used to classify W into clusters. Our approach is based on the binary reduced row echelon form of data matrix, combined with an iterative scheme based on a non-linear approximation method. We prove that, in absence of noise, our approach can find the number of subspaces, their dimensions, and an orthonormal basis for each subspace Si. We provide a comprehensive analysis of our theory and determine its limitations and strengths in presence of outliers and noise

Application of Subspace Clustering in DNA Sequence Analysis

Identification and clustering of orthologous genes plays an important role in developing evolutionary models such as validating convergent and divergent phylogeny and predicting functional proteins in newly sequenced species of unverified nucleotide protein mappings. Here, we introduce an application of subspace clustering as applied to orthologous gene sequences and discuss the initial results. The working hypothesis is based upon the concept that genetic changes between nucleotide sequences coding for proteins among selected species and groups may lie within a union of subspaces for clusters of the orthologous groups. Estimates for the subspace dimensions were computed for a small population sample. A series of experiments was performed to cluster randomly selected sequences. The experimental design allows for both false positives and false negatives, and estimates for the statistical significance are provided. The clustering results are consistent with the main hypothesis. A simple random mutation binary tree model is used to simulate speciation events that show the interdependence of the subspace rank versus time and mutation rates. The simple mutation model is found to be largely consistent with the observed subspace clustering singular value results. Our study indicates that the subspace clustering method may be applied in orthology analysis

Radius of curvature and location estimation of cylindrical objects with sonar using a multi-sensor configuration

Ankara : Department of Electrical and Electronics Engineering and the Institute of Engineering and Sciences of Bilkent University, 1997.Thesis (Master's) -- Bilkent University, 1997.Includes bibliographical references leaves 130-133.Despite their limitations, sonar sensors are very popular in time-of-flight measuring systems since they are inexpensive and convenient. One of the most important limitations of sonar is its low angular resolution. An adjustable multi-sonar configuration consisting of three transmitter/receiver ultrasonic transducers is used to improve the resolution. The radius of curvature estimation of cylindrical objects is accomplished with this configuration. Two different ways of rotating the transducers are considered. First, the sensors are rotated around their joints. Second, the sensors are rotated around their centers. Also, two methods of tirne-of-flight estimation are implemented which are thresholding and curve-fitting. Sensitivity analysis of the radius of curvature with respect to some important parameters is made. The bias-variance combinations of both estimators are compared to the Cramer-Rao lower bound. Theory and simulations are verified by experimental data from real sonar systems. Data is smoothed by extended Kalman filtering. Rotating around the center works better than rotating around the joint. Curve-fitting method is shown to be better than thresholding method both in the absence and presence of noise. The best results are obtained w'hen the sensors are rotated around their centers and the curve-fitting method is used to estimate the time- of-flight. There is about 30% improvement in the absence of noise and 50% improvement in the presence of noise.Sekmen, Ali ŞafakM.S

A deep learning approach for motion segment estimation for pipe leak detection robot

The trajectory motion of a robot can be a valuable information to estimate the localization of an autonomous robotic system, especially in a very dynamic but structurally-known environments like water pipes where the sensor readings are not reliable. The main focus of this research is to estimate the location of meso-scale robots using a deep-learning-based motion trajectory segment detection system from recorded sensory measurements while the robot travels through a pipe system. The idea is based on the classification of the motion measurements, acquired by inertial measurement unit (IMU), by exploiting the deep learning approach. Proposed idea and utilized methodology are explained in the related sections and it is observed that convolutional neural network approach is quite powerful to overcome the unreliability of IMU data

Mathematical and Machine Learning Approaches for Classification of Protein Secondary Structure Elements from Cα Coordinates

Determining Secondary Structure Elements (SSEs) for any protein is crucial as an intermediate step for experimental tertiary structure determination. SSEs are identified using popular tools such as DSSP and STRIDE. These tools use atomic information to locate hydrogen bonds to identify SSEs. When some spatial atomic details are missing, locating SSEs becomes a hinder. To address the problem, when some atomic information is missing, three approaches for classifying SSE types using Ca atoms in protein chains were developed: (1) a mathematical approach, (2) a deep learning approach, and (3) an ensemble of five machine learning models. The proposed methods were compared against each other and with a state-of-the-art approach, PCASSO

Robust feature space separation for deep convolutional neural network training

This paper introduces two deep convolutional neural network training techniques that lead to more robust feature subspace separation in comparison to traditional training. Assume that dataset has M labels. The first method creates M deep convolutional neural networks called {DCNNi}M i=1 . Each of the networks DCNNi is composed of a convolutional neural network ( CNNi ) and a fully connected neural network ( FCNNi ). In training, a set of projection matrices are created and adaptively updated as representations for feature subspaces {S i}M i=1 . A rejection value is computed for each training based on its projections on feature subspaces. Each FCNNi acts as a binary classifier with a cost function whose main parameter is rejection values. A threshold value ti is determined for ith network DCNNi . A testing strategy utilizing {ti}M i=1 is also introduced. The second method creates a single DCNN and it computes a cost function whose parameters depend on subspace separations using the geodesic distance on the Grasmannian manifold of subspaces S i and the sum of all remaining subspaces {S j}M j=1,j≠i . The proposed methods are tested using multiple network topologies. It is shown that while the first method works better for smaller networks, the second method performs better for complex architectures